(C) Copyright 1996 Texas Instruments Incorporated. A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
This invention generally relates to audio-visual systems and integrated circuits used therein, and more particularly, to improved audio-visual systems and improved integrated circuits used therein.
Currently, digital television DSS/DVB signals are transmitted in an MPEG format. In the MPEG format, the signals are transmitted in the form of data packets, with each packet including a useful signal or data and a header signal or data containing information about the contents of the useful signal or data.
Such a digital signal generally consists of packets of various types, such as audio data packets, video data packets and packets containing information pertaining to the program transmitted.
In general, the decoding of such digital signals takes place in a decoder in a reception station, which identifies and selects, in the incoming signals, desired audio and video data packets and then decodes these packets to form data trains, of the audio and video types, respectively. The audio data trains are decoded by means of an audio decoder for forming an analog acoustic signal. Similarly, the video data trains are used to form an image as well as chrominance and luminance signals.
There are known devices for the identification of packets. These devices extract a piece of data from each header signal, which is representative of the type of the corresponding useful signal. They include a means for storing reference data in a memory, at addresses each corresponding to one packet type, and a means for comparing the piece of data extracted from each header signal with said reference data (stored in the memory) then delivering to a data processing unit for further processing, an address signal indicating the nature of the corresponding packet. The processing unit then selects the identified packets for decoding and for forming corresponding data trains.
For this type of identification device, the comparison between the piece of data extracted from the header signal and the reference data stored in memory is conducted successively; that is, at a transition of a synchronization clock, an extracted piece of data is compared to a reference piece of data.
However, since the transmission rate of the packets is very high, the packet being identified must be stored, for example, in a memory of the FIFO type, associated with a piloting circuit and then further processed by the data processing unit.
Consequently, this type of identification device is relatively slow, requires a large number of components, and requires a large amount of memory and/or local storage buffers.
International standardization committees have been working on the specification of the coding methods and transmission formats for several compression algorithms to facilitate world wide interchange of digitally encoded audiovisual data. The Joint Photographic experts Group (JPEG) of the International Standards Organization (ISO) specified an algorithm for compression of still images. The ITU (formerly CCITT) proposed the H.261 standard for video telephony and video conference. The Motion Pictures Experts Group (MPEG) of ISO specified a first standard, MPEG-1, which is used for interactive video and provides a picture quality comparable to VCR quality. MPEG has also specified a second standard, MPEG-2, which provides audiovisual quality of both broadcast TV and HDTV. Because of the wide field of applications MPEG-2 is a family of standards with different profiles and levels.
The JPEG coding scheme could be in principal also used for coding of images sequences, sometimes described as motion JPEG. However, this intraframe coding is not very efficient because the redundancy between successive frames is not exploited. The redundancy between succeeding frames can be reduced by predictive coding. The simplest predictive coding is differential interframe coding where the difference between a current pixel of the present frame and the corresponding pixel of the previous frame is quantized, coded and transmitted. To perform such interframe prediction a frame memory for storing one or more frames is required to allow for this pixel by pixel comparison. Higher efficiency than the simple differential interframe coding can be achieved by a combination of discrete cosine transform (DCT) and interframe prediction. For so-called hybrid coding the interframe difference, which is similar to JPEG, is obtained, DCT coded and then transmitted. In order to have the same prediction at both the receiver and transmitter the decoder is incorporated into the coder. This results in a special feedback structure at the transmitter which avoids coder-decoder divergence.
Variable word length coding results in a variable bit rate which depends on image content, sequence change, etc. Transmission of the coded information over a constant rate channel requires a FIFO buffer at the output to smooth the data rate. The average video rate has to be adjusted to the constant channel rate. This is performed by controlling the quantizer according to the buffer content. If the buffer is nearly full, the quantization is made more sever and thus the coded bitrate is reduced. Conversely, if the buffer is nearly empty, the quantization is relaxed.
In general, the MPEG coding use a special predictive coding strategy. The coding starts with a frame which is not differentially coded; it is called an Intraframe (I). Then prediction is performed for coding one frame out of every M frames. This allows computation of a series of predicted frames (P), while xe2x80x9cskippingxe2x80x9d M-1 frames between coded frames. Finally, the xe2x80x9cskippedxe2x80x9d frames are coded in either a forward prediction mode, backward prediction mode, or bi-directional prediction mode. These frames are called bi-directionally interpolated (B) frames. The most efficient prediction mode, in terms of bitrate, is determined by the encoder and its selected mode is associated with the coded data. Thus the decoder can perform the necessary operations in order to reconstruct the image sequence. A main difference between MPEG-1 and MPEG-2 is that MPEG-1 has been optimized for non-interlaced (progressive) format while MPEG-2 is a generic standard for both interlaced and progressive formats. Thus, MPEG-2 includes more sophisticated prediction schemes.
In more detail, motion pictures are provided at thirty frames per second to create the illusion of continuous motion. Since each picture is made up of thousands of pixels, the amount of storage necessary for storing even a short motion sequence is enormous. As higher and higher definitions are desired, the number of pixels in each picture grows also. This means that the frame memory used to store each picture for interframe prediction also grows; current MPEG systems use about 16 megabits (MB) of reference memory for this function. Fortunately, lossy compression techniques have been developed to achieve very high data compression without loss of perceived picture quality by taking advantage of special properties of the human visual system. (A lossy compression technique involves discarding information not essential to achieve the target picture quality to the human visual system). An MPEG decoder is then required to reconstruct in real time or nearly real time every pixel of the stored motion sequence; current MPEG decoders use at least about 16 MB of frame memory for reconstruction of frames using the encoded interframe prediction data.
The MPEG standard specifies both the coded digital representation of video signal for the storage media, and the method for decoding to achieve compatibility between compression and decompression equipment. The standard supports normal speed playback, as well as other play modes of color motion pictures, and reproduction of still pictures. The standard covers the common 525- and 625-line television, personal computer and workstation display formats. The MPEG-1 standard is intended for equipment supporting continuous transfer rate of up to 1.5 Mbits per second, such as compact disks, digital audio tapes, or magnetic hard disks. The MPEG-2 standard supports bit rates from 4 Mbits/sec (Mbits) to 15 Mbits and is targeted for equipment that complies with the International Radio Consultative Committee (CCIR) recommendation 601 (CCIR-601). The MPEG standard is intended to support picture frames at a rate between 24 Hz and 30 Hz. ISO-11171 entitled Coding for Moving Pictures and Associated Audio for digital storage medium at 1.5 Mbit/s, provides the details of the MPEG-1 standard. ISO-13838 entitled Generic Coding of Moving Pictures and Associated Audio provides the details of the MPEG-2 standard.
Under the MPEG standard, the picture frame is divided into a series of xe2x80x9cMacroblock slicesxe2x80x9d (MBS), each MBS containing a number of picture areas (called xe2x80x9cmacroblocksxe2x80x9d) each covering an area of 16xc3x9716 pixels. Each of these picture areas is represented by one or more 8xc3x978 matrices which elements are the spatial luminance and chrominance values. In one representation (4:2:2) of the macroblock, a luminance value (Y type) is provided for every pixel in the 16xc3x9716 pixels picture area (in four 8xc3x978 xe2x80x9cYxe2x80x9d matrices), and chrominance values of the U and V (i.e., blue and red chrominance) types, each covering the same 16xc3x9716 picture area, are respectively provided in two 8xc3x978 xe2x80x9cUxe2x80x9d and two 8xc3x978 xe2x80x9cVxe2x80x9d matrices. That is, each 8xc3x978 U or V matrix covers an area of 8xc3x9716 pixels. In another representation (4:2:0), a luminance value is provided for every pixel in the 16xc3x9716 pixels picture area, and one 8xc3x978 matrix for each of the U and V types is provided to represent the chrominance values of the 16xc3x9716 pixels picture area. A group of four continuous pixels in a 2xc3x972 configuration is called a xe2x80x9cquad pixelxe2x80x9d; hence, the macroblock can also be thought of as comprising 64 quad pixels in an 8xc3x978 configuration.
The MPEG standard adopts a model of compression and decompression. Initially, interframe redundancy is first removed from the color motion picture frames. To achieve interframe redundancy removal, each frame is designated either xe2x80x9cintraxe2x80x9d xe2x80x9cpredictedxe2x80x9d or xe2x80x9cinterpolatedxe2x80x9d for coding purpose. Intraframes are least frequently provided, the predicted frames are provided more frequently than the intraframes, and all the remaining frames are interpolated frames. The values of every pixel in an intraframe (xe2x80x9cI-picturexe2x80x9d) is independently provided. In a prediction frame (xe2x80x9cP-picturexe2x80x9d), only the incremental changes in pixel values from the last I-picture or P-picture are coded. In an interpolation frame (xe2x80x9cB-picturexe2x80x9d), the pixel values are coded with respect to both an earlier frame and a later frame. Again, large (16 MB) frame or reference memories are required to store frames of video to allow for this type of coding.
The MPEG standard does not require frames to be stored in strict time sequence, so that the intraframe from which a predicted frame is coded can be provided in the picture sequence either earlier or later in time from the predicted frame. By coding frames incrementally, using predicted and interpolated frames, much interframe redundancy can be eliminated which results in tremendous savings in storage requirements. Further, motion of an entire macroblock can be coded by a motion vector, rather than at the pixel level, thereby providing further data compression.
The next steps in compression under the MPEG standard remove intraframe redundancy. In the first step, a 2-dimensional discrete cosine transform (DCT) is performed on each of the 8xc3x978 values matrices to map the spatial luminance or chrominance values into the frequency domain.
Next, a process called xe2x80x9cquantizationxe2x80x9d weights each element of the 8xc3x978 matrix in accordance with its chrominance or luminance type and its frequency. In an I-picture, the quantization weights are intended to reduce to one many high frequency components to which the human eye is not sensitive. In P- and B- pictures, which contain mostly higher frequency components, the weights are not related to visual perception. Having created many zero elements in the 8xc3x978 matrix, each matrix can now be represented without information loss as an ordered list of a xe2x80x9cDCxe2x80x9d value, and alternating pairs of a non-zero xe2x80x9cACxe2x80x9d value and a length of zero elements following the non-zero value. The list is ordered such that the elements of the matrix are presented as if the matrix is read in a zigzag manner (i.e., the elements of a matrix A are read in the order A00, A01, A10, A20, A11, A02, etc.). The representation is space efficient because zero elements are not represented individually.
Finally, an entropy encoding scheme is used to further compress the representations of the DC block coefficients and the AC value-run length pairs using variable length codes. Under the entropy encoding scheme, the more frequently occurring symbols are represented by shorter codes. Further efficiency in storage is thereby achieved.
In decompression, under MPEG, the processes of entropy encoding, quantization and DCT are reversed. The final step, called xe2x80x9cabsolute pixel generationxe2x80x9d, provides the actual pixels for reproduction, in accordance to the play mode (forward, reverse, slow motion e.g.), and the physical dimensions and attributes of the display used. Again, large (16 MB) frame or reference memories are required to store frames of video to allow for this type of reproduction.
Since the steps involved in compression (coding) and decompression (decoding), such as illustrated for the MPEG standard discussed above, are very computationally intensive and require large amounts of memory, for such a compression scheme to be practical and widely accepted, the decompression processor must be designed to provide decompression in real time, and allow economical implementation using today""s computer or integrated circuit technology.
The purpose of the present invention is to overcome these short-comings and drawbacks. Improvements in circuits, integrated circuit devices, computer systems of all types, and methods to address all the just-mentioned challenges, among others, are desirable, as described herein.
Generally, and in one form of the present invention, an improved audio-visual circuit is provided that includes a transport packet parsing circuit for receiving a transport data packet stream, a CPU circuit for initializing said integrated circuit and for processing portions of said data packet stream, a ROM circuit for storing data, a RAM circuit for storing data, an audio decoder circuit for decoding audio portions of said data packet stream, a video decoder circuit for decoding video portions of said data packet stream, an NTSC/PAL encoding circuit for encoding video portions of said data packet stream, an OSD coprocessor circuit for processing OSD portions of said data packets, a traffic controller circuit moving portions of said data packet stream between portions of said integrated circuit, an extension bus interface circuit, a P1394 interface circuit, a communication coprocessors circuit, an address bus connected to said circuits, and a data bus connected to said circuits.
The present invention provides a fully functional decoder using a single 16 Mbit external SDRAM.
The present invention provides a decoder that accepts transport bitstreams up to 40 Mbits per second.
The present invention provides an on-chip DES module for descrambling.
The present invention provides a video decoder that decodes MPEG-1 and MPEG-2 Main Profile and Main Level bitstreams.
The present invention provides an audio decoder that decodes MPEG-1 Layer I and II and MPEG-2 Multichannel bitstreams.
The present invention provides an audio output in both PCM and SPDIF formats.
The present invention provides an OSD processor that enables mixture of OSD and video data with transparent BitBLT hardware that accelerates memory block move.
The present invention provides a 32/16 bit ARM/Thumb processor that removes the need of another CPU in the set-top box.
The present invention provides a firmware that controls device operation and provides application access to hardware resources.
The present invention provides an on-chip NTSC/PAL encoder that incorporates Closed Caption and Video Aspect Ratio Identification Signal encoding and the MacroVision logic for anti-taping protection.
The present invention provides an analog Y, C, and Composite video outputs with 9-bit precision.
The present invention provides an internally or externally generated sync signals.
The present invention provides a digital video component output that also contains Aspect Ratio ID.
The present invention provides an on-chip SDRAM controller for 16, 20, 24, or 32 Mbit SDRAM.
The present invention provides a general purpose 16-bit extension bus.
The present invention provides a 1394 interface that allows connection to external 1394 devices.
The present invention provides two 2-wire UART data ports.
The present invention provides a Smart Card interface.
The present invention provides an I2C master/slave interface.
The present invention provides one IR, one SIRCSI, and one RF input data port.
The present invention provides two general purpose I/O pins.
The present invention provides a JTAG interface.
The present invention provides a 3.3 volt device with some 5 volt tolerant pins for interfacing to the 5 volt devices.
It is an object of the present invention to provide a fully functional decoder using a single 16 Mbit external SDRAM.
It is an object of the present invention to provide a decoder that accepts transport bitstreams up to 40 Mbits per second.
It is an object of the present invention to provide an on-chip DES module for descrambling.
It is an object of the present invention to provide a video decoder that decodes MPEG-1 and MPEG-2 Main Profile and Main Level bitstreams.
It is an object of the present invention to provide an audio decoder that decodes MPEG-1 Layer I and II and MPEG-2 Multichannel bitstreams.
It is an object of the present invention to provide an audio output in both PCM and SPDIF formats.
It is an object of the present invention to provide an OSD processor that enables mixture of OSD and video data with transparent BitBLT hardware that accelerates memory block move.
It is an object of the present invention to provide a 32/16 bit ARM/Thumb processor that removes the need of another CPU in the set-top box.
It is an object of the present invention to provide a firmware that controls device operation and provides application access to hardware resources.
It is an object of the present invention to provide an on-chip NTSC/PAL encoder that incorporates Closed Caption and Video Aspect Ratio Identification Signal encoding and the MacroVision logic for anti-taping protection.
It is an object of the present invention to provide an analog Y, C, and Composite video outputs with 9-bit precision.
It is an object of the present invention to provide an internally or externally generated sync signals.
It is an object of the present invention to provide a digital video component output that also contains Aspect Ratio ID.
It is an object of the present invention to provide an on-chip SDRAM controller for 16, 20, 24, or 32 Mbit SDRAM.
It is an object of the present invention to provide a general purpose 16-bit extension bus.
It is an object of the present invention to provide a 1394 interface that allows connection to external 1394 devices.
It is an object of the present invention to provide two 2-wire UART data ports.
It is an object of the present invention to provide a Smart Card interface.
It is an object of the present invention to provide an I2C master/slave interface.
It is an object of the present invention to provide one IR, one SIRCSI, and one RF input data port.
It is an object of the present invention to provide two general purpose I/O pins.
It is an object of the present invention to provide a JTAG interface.
It is an object of the present invention to provide a 3.3 volt device with some 5 volt tolerant pins for interfacing to the 5 volt devices.